Controller-free interactive systems, such as gaming systems, may be controlled at least partially by natural movements. In some examples, such systems may employ a depth sensor, or other suitable sensor, to estimate motion of a user and translate the estimated motions into commands to a console of the system. However, in estimating the motions of a user, such systems may only estimate major joints of the user, e.g., via skeleton estimation, and may lack the ability to detect subtle gestures.